There’s not a company on the planet that isn’t generating and collecting data in some way. Today alone, the world will generate trillions upon trillions of bits of information. But there’s a problem with gathering all that data: It’s way too much for companies and their chief data officers (CDOs) to handle — and way too much for people to comprehend.
As well, much of what’s collected is called dark data, information that’s collected but never used. In our experience, dark data makes up most of the information companies collect. Unstructured data — videos, images, emails and other points that can’t be inserted into a spreadsheet — falls into this category, but businesses also collect a lot of information that for one reason or another never gets analyzed.
What’s exciting is that while a lot of valuable data is considered dark, when brought to light it can reveal real insights into the wants and needs of your organization, including around your customers and employees, the quality of your assets and manufacturing, and the risks your brand faces on social media.
CDOs need to get a handle on their dark data now, as their companies are gathering increasing amounts of information every day. Businesses are putting data-collecting sensors on pretty much everything — industrial equipment, agricultural fields, office building walls and people’s wrists. They’re using drones to measure climate impacts and snap pictures of assets, while artificial intelligence (AI) technology is finding data points in minutes rather than the days, weeks or years it used to take to uncover.
While companies don’t need to analyze every number or statistic that comes their way, they also shouldn’t continue keeping all this data in the dark.
A lot of this is potentially valuable information that can be used to increase productivity and boost growth. And not knowing what’s there can pose significant risks.
Say a utility has sensors on its equipment to detect gas leaks and a safety incident occurs. Now suppose that there was information hidden in the dark data those sensors collected that could have been used to predict and prevent the incident, but the utility never looked at that data. Having the data available but unused could bring lawsuits or a public relations nightmare.
Or take a bank dealing with a fraud case. An analysis of its dark data might have revealed red flags that would have prevented the breach. Worse, regulators — who now have a much better understanding of the data companies collect — might be unimpressed with a bank defending itself with “how could we have known that was coming?” Regulators expect businesses to use all the tools they have to prevent fraud and generally crack down if they see something that a company missed.
How can companies make their data more visible? Here are some ideas.
The more data that your company can analyze, the better decisions you’ll make. While it’s important to not get overloaded with information — keep going back to your business objectives and the risks you’re accounting for to determine what to observe — you should still be aware of what you’re collecting.
As companies become even more data-focused, and as new tools emerge to help people analyze their unstructured information, it’s only a matter of time before your dark data gets out in the open. The companies that use their data to their advantage can be the ones that get ahead.
Empower yourself with smarter insights that drive better, faster, more confident decisions.